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Abstract 

The validity of conventional standardized tests for personality 
assessment of ethnic minority children has been widely 
criticized, both in the public domain and research literature 
in the United States. On conventional projective personality 
tests (e.g., TAT, CAT), Hispanic children have been evaluated 
as less verbally fluent tnan their nonminority counterparts, 
which impugns the validity of such techniques. This paper 
presents psychometric research conducted on TEMAS (meaning 
"themes" in Spanish), a thematic apperception test depicting 
ethnically pluralistic characters interacting in familiar 
settings. This test consists of 23 chromatic pictures 
posing an^-ithetical dilemmas which the examinee must resolve 
in telling a story about each picture. Based upon the 
adaptiveness of examinees* themes, personality functioning 
is rated in terms of constructs such as delay of gratification, 
achievement motivation, reality testing, and self concept. 
The TEMAS test has been standardized with Hispanic children 
(grades K-6) in New York and Puerto Rico. Preliminary 
studies of the New York population indicated that Hispanic 
children were more verbally fluent on the TEMAS than the 
TAT, and that TEMAS pictures evoked stable themes over 
a 4-month interval. Other studies established internal 
consistency and inter-clinician reliability in ratino TEMAS 
protocols. Rudimentary evidence of concurrent validity 
was established, as well as Reliability ani validity studies 



conducted in Puerto Rico are compared with studies conducted 

in New York. Results are discussed in terms of culturally-referenced 

test development and interpretation of personality functioning. 
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Numerous clinicians and researchers alike have argued 
that projective techniques provide a more comprehensive 
and clinically meaningful assessment of personality functioning 
than "paper and pencil" tests (e.g.r Rapportr 1942), and 
also obviate attendant problems with validation of personality 
inventories for cross-cultural research (Padilla, 1979, 
1973). Traditionally, projective tests have fared well 
in clinical settings, but have presented a host of problems 
with respect to psychometric rigor (Murstein, 1963). In 
rec *t years, however, there has been a resurgence of interest 
in developing clinically useful and psychometrically acceptable 
projective techniques (Exner, 1978; Holtzman, Thorpe, Swartz, 
Herron, 1961; Sobel, 1981). 

A prominent issue of widespread debate in the United 
States is the validity of evaluating ethnic minorities 
with instruments that have been standardized on white, 
:niddle-class norm groups (Olmedo, 1981; Padilla, 1979). 
While some attempts have been made to develop minority-oriented 
cognitive and personality tests for blacks (Thompson, 1949; 
Williams, 1977; 1982), as well as Hispanics (Ortiz & Ball, 
1977; Struthers & DeAvila, 1977), unfortunately such tests 
have not weathered critical psychometric scrutiny (Oakland 
1977). 

A decade ago, Padilla and Ruiz (1973) reported that 
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very little research had focused on the psychological assessment 
of Hispanics in particular with projective techniques r 
and neglect of this research topic persists today (Rogler 
et al.r 1983). The use of traditional thematic apperception 
tests r such as the TAT and CATr has led to the evaluation 
of both Hispanic and black children as less verbally fluent ♦ 
and .Tiore psychopathological than their nonminority counterparts 
(Ames & August r 1966; Booth r 1966). Yet these assessment 
practices are ubiquitous despite acknowledgement that the 
validity of projective techniques can be impugned with 
verbally inarticulate examinees (Anderson & Anderson^ 1955). 
Hence there is a compelling need to develop psychological 
tests for reliable and valid diagnosis and personality 
assessment of ethnic racial and linguistic minority children 
(Padilla, 1979). 

The problem of developing valid and reliable culturally-sensitive 
psychiatric evaluations and psychometric instruments is 
a serious one in a multicultural and pluralistic American 
society and is compounded when we endeavor to establish 
valid psychiatric diagnoses and assessment in cross-cultural 
settings (Good and Del Vecchio-Good^ 1986; Malgady^ Rogler, 
and Costantino^ 1987). 
Development of lEMAS 

Based upon these considerations ^ the TEMAS thematic 
apperception test was developed with structured ^ familiar 
and culturally relevant stimuli. There are two parallel 
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sets of TEMASr minority' and nonminority versions (Costantino, 

1986 )• TEMAS testr minority version r which consists of 

23 chromatic pictures depicting minority characters in 

urban setting r embodies following features: 1) use of 

structured stimuli and diminished ambiguity to pull for 

specifier covert behaviors and internal dialogues; 2) use 

of chromatically attractive r ethnically relevant and contemporary 

stimuli to elicit diagnostically meaningful stories; 3) 

representation of both negative and positive intrapersonal 

and interpersonal functions in the form of conflicts or 

dilemas which require a resolution; 4) interaction of cognitive r 

affective r and intrapersonal functions; and 5) use of an 

objective scoring system in analyzing TEMAS stories ^ whu.jh 

assesses both the structure and content of the stories 

(Costantinor 1978 ) . 

The theoretical framework and underlying the construction 
of TEMAS test is based generically in the traditional egopsychological 
theory as represented in Bellak (1971)^ theoretical reconsideration 
of the original Thematic Apperception Test (Murray^ 1943), 
in the dynamic Motivational Psychology proposed initially 
by McClelland and Atkinson (1953) and more recently articulated 
by Atkinson (1981), and in the cognitive and social learning 
theories (Bandura 1977, Piaget 1971). Accordingly, p^.rsonality 
structure is conceived within this framework as a constellation 
of latent motives or ego functions which are defined as 
internalized dispositions that interact with environmental 
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stimuli to determine overt behavior in specific situations. 
Since these dispositions are not directly accessible to 
clinical inquiry, projective techniques prove to be useful 
tools for probing beneath the surface structure of "phenotype" 
of personality, thereby arousing the latent motives embedded 
in the personality "genotype" (1981). Access to the genotypic 
structure of personality is accomplished by projective 
test stimuli that provoke imaginative or fantasy processes 
in telling stories about the stimuli. Therefore, motivational 
theorists maintain that projective tests assess relatively 
stable individual differences in the strength of underlying 
motives, which are behaviorally expressed in thematic content, 
and also reveal individual differences in the antecedent 
developmental histories of examinees. Based upon this 
reasoning, Atkinson (1981) contends that the analysis of 
thematic content has a "more solid theoretical foundation 
than ever before and must... be considered the most promising 
and virtually untapped resource for future stud/ of personality. 
Furthermore, developmental and learning theorists (Bandura, 
1977; Piaget, 1962; 1971) have suggested that personality 
functions are acquired through modeling, and then psychologically 
instantiated through ve^ :>al and imaginal processes. Hence, 
advocates of projective techniques conclude that since 
personality functions are acquired within a given a sociocultural 
context, they are readily transferred to the testing situation 
and projected into thematic content when the stimuli are 



similar to the circumstances in which these functions were 
originally learned (Auld, 1954), 

Based this theoretical rationale for the thematic 
apperception technique in studying personalityr TEMAS was 
developed in response to the concerns^ raised earlier^ 
about assessment of ethnic and linguistic minority children. 
Consequentlyr the traditional TAT was modified in TEMAS 
by presenting ethnically pluralistic characters in familiar 
settings^ while engaging in antithetical situations representing 
common problematic circumstances in urban life (Costantino^ 
1978 )• Thus TEMAS is a an attempt to increase the ethnocultural 
relevance of projective stimuli to minority children^ and 
to present familiar scenes associated with life experiences 
in inner-city settings. The rationale for these and other 
departures (e.g.r reduced ambiquity^ chromaticity ) from 
traditional thematic apperception technique is based on 
empirical research conducted with the TATr and is summarized 
below. 

The presentation of culturally relevant and familiar 
stiknuli in projective tests was explored on by Thompson 
(1949) who developed a black TAT based upon the assumption 
that similarity between the stimulus and the examinee promotes 
identification with the characters in the pictures^ and^ 
therefore^ provokes greater verbal fluency and self-disclosure. 
Traditionally^ projective techniques have embraced the 
psychodynamic notion that ambiguous stimuli bypass the 
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examinee's ego defenses, thereby allowing latent psychological 
conflict to be more freely expressed (Murstein, 1963). 
However, TEMAS was conceived following more recent thinking 
that diminished ambiquity and increased structure in projective 
stimuli facilitate verbal fluency and yield a more focused 
understanding of the examinee's personality functioning 
(Eps^^ein, 1966). That is, when projective stimuli are 
structured to "pull" specific personality functions (unlike 
ambiguous stimuli), the diagnostician may achieve a more 
reliable and valid clinical interpretation of thematic 
content. 

Both clinicians and researchers alike acknowledge 
that color has strong impact on the perception of Rorschach 
cards, (Murstein, 1963) and that integration of color and 
from is considered a sign of emotional growth and cognitive 
organization (Siipola, 1950). During the 1950 's several 
studies documented that chromatic TAT pictures more accurately 
discriminated between clinical and control subjects that 
achromatic pictures and also enhanced verbal fluency (Brackbill, 
1951; Thompson and Backrach, 1951) Murstein (1963) explained 
that color facilitates differentiation between thematic 
responses of psychiatric and normal examinees, apparently 
since achromaticity reinforces sadness as an effective 
response to TAT pictures. Thus, based upon this evidence, 
TEMAS pictures were developed in life-like color. 

The representation of psychological conflict in TEMAS 
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pictures was based on the methodology of Kohlberg (1976) 

who suggested that moral judgement develops only when the 

child is able to understand reversible operations which 

are antithetical in nature. Hence, Kohlberg developed 

a series of stories portraying anthithetical moral dilemmas 

in order to assess the moral devalopment of children. 

Similarly, TEMAS pictures portray a split scene showing 

psychological dilemmas which require a resolution. Like 

Kohlberg •s moral dilemmas, TEMAS examinees must resolve 

the antithetihcal situations portrayed in the split pictures; 

and the examining clinician evaluates the adaptiveness 

of their resolution of the conflict. The antithetical 

situation depicted in TEMAS pictures were designed to evoke 

disclosure of specific personality functions which are 

prominent in personality theory and also are key diagnostic 

indices of psychopathology : Interpersonal relations, aggres'-^ion, 

anxiety/depression, achievement motivation, delay of gratification, 

self concept of competence, self /sexual identity, moral 

judgement, and reality testing (Bellak, Hurvich and Gediman, 

1973? Costantino, 1978). 

An initial set of TEMAS pictures was drawn by a professional 
artist, who worked closely with the test author (Costantino. 
1978) in an attempt to pictorially represent the psychosocial 
situation described to him. In some situations characters 
were shown engaging in internal dialogues, such as a child 
in front of a piggybank with a coin in hand imagining buying 
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an ice cream cone immediately, or saving money to buy a 
bicycle in the future. This picture represent ambivalence 
associated with ability to delay gratification (Mischel, 
1966). Other pictures, such as 9B and 9G (see TAble 1,) 
are pictorial representations of the poem by Robert Frost, 
"The Road Not Taken." These pictures depict a boy or girl 
at a crossroad in a forest undecided to take the road already 
taken by his or her peers, or to take the road which no 
one has taken (evoking interpersonal relationships and 
anxiety feelings). A brief description of the final set 
of 23 pictures resulting from pilot studies is presented 
in Table 1, which also denotes the personality functions 
pulled by each picture. As Table 1 illustrates, the TEMAS 
pictures embody a wide variety of problematic life situation 
and experiences in inner-city, impoverished environments. 
Table 1 descriptions include intraf amilial scenes within 
the home, solitary dream-like and fantasy states, street 
scenes involving peers and adults, sports activities, and 
situations occurring in school settings. The antithetical 
situations portrayed in the pictures accommodate a balance 
of positive or negative feelings to be projected in thematic 
content, manifested behaviorally in either adaptive or 
maladaptive resolutions of the dilemmas presented. These 
\:opics are structured to pull themes expressive of varying 
degrees of psychopathology ranging from severe pathology 
(e.g., morbidityr suicide, depression, gender confusion. 
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impulsivityr isolationr delusion) to lack of pathology 
(Costantino^ Malgadyr & Rogler^ 1987-in press). Some example 
of underlying motives revealed in thematic content which 
vary in level of expressed psychopathology are provided 
*-he scoring procedures. 

Table 1 

Picture No. Description 

*1B/G. A mother is giving a command to her son/daugher 

while the father is in the background. Peers 
are urging the boy/girl to play basketball/ jump 
rope with them (IPr DG) 

2. A father is watching television and drinking, 
a sen and daugher are standing behind him. 

The mother is carrying her infant and vacummingi 
while two additional daughters and a son 
are by e mother side (IPr DG) 

3. A father is telling his son who wants to 
watch T.V. to do his homework. Three sibling^ 
are watching television. (IP^ AMr DG) 

4. An angry father is threatening the mother 
while a young woman lies in bed covering 
her face with her hands. Two boys and girls 
are clinging to the mother (IP^ AGG^ ANXr 
MJ) 

5. An adolescent is sleeping in bed and dreaming 
of a picnic with a female^ or of a dark figure 
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entering a bedroom at night. (IP^ AGG) 
6. A boy and a girl dress up in adult clothes 

in the attic, while they look nostalgically 
at a cradle and some baby toys, (IP, 31) 
*7. Angry mother is watching her son and daughter 

arguing over a broken lamp. (IP, AGG, MJ) 
8. A male teacher is with a group of attentive 

students. A female teacher/principal shows 
a mother and a father a broken window with 
a boy and girl, behind the parent. (IP, AGG, 
AM, MJ) 

9B/G. A boy/girl with outstretched arms is standing 

at the junction of two roads in the forest. 
Friends call to him/her to join them walking 
on the right hand road. A lonely road is 
on the left. (IP, ANX) 

*10B/G. A boy/girl is holding money standing in front 

of a piggybank, imagining him/herself looking 
at a bicycle in a shop window or buying an 
ice cream cone. (DG) 

11. A mother is carrying a bag of groceries with 

her son and daughter helping her. A woman 
is trying to protect herself from two boys 
and a girl who are stealing groceries from 
her bag. (IP, AGG, MJ) 

12B/G. A group of boys/girls are cooperating in 
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the repair of a bicycle. A group of boys/girls 
are fighting (IP^ AGG) 
13B/G. A boy/girl is standing in front of a bathroom 

mirror imagining the reflection of his/her 
parents in the mirror. (IP^ sCCr SI) 
*14B/G. A boy/girl is studying in his/her room. 

A group of boys and girls are listening to 
music in the living room. (IP, s^Cr SI) 
A policeman is giving an award to a group 
of PAL baseball player. A policeman is arresting 
a group of boys and girls who broke a window 
and stole merchandise. (IP^ AGG^ AM^ MJ) 
16. A boy is climbing up a rope in a gym and 

girls are jumping over a wooden horse. A 
group of two and two girls on one side of 
the picture are expressing encouragement 
and admiration. A group of two boys and 
two girls on the other side of the picture 
are expressing fearfulness. (IP^ AM^ SCO 
17B. A boy/girl is studying and daydreaming about 

him/herself receiving an "A" or "F" from 
the teacher. (ANS^ AM^ SCO 
18B/G. A boy/girl studying and daydreaming about 

becoming an actor/act ^^rss^ a doctor, or a 
drunk/bag lady. (AM, SCO 
19B/G. A boy/girl is in a window imagining him/herself 



12 



being saved from a building in flames by 

a fireman or by Superman/Wonder Woman, (ANX, 

RT) 

20. An adolescent in bed is dreaming of a scene 

showing a horse trapped on a hill over a 
river, or a path leading to a castle. (ANX) 

*21. An adolescent in bed is dreaming of a friendly 

monster eating something or of a monster 
making threats. (AGG, ANX RT) 

*22B/G. A boy/girl is standing in .ront of a bathroom 

minor, imagining his/her face ref ^ted 
in the mirror with attributes of boch sexes. 

23B/G. A boy/girl is rejected by his/her parents, 

and imagines running away from home and living 
alone, or standing on a bridge thinking of 
committing suicide. (IP, ANX) (Costantino, 1986). 

Note: (*)Denotes a card belinging to the short 

form. The designations and "B" and "G" refer 
to sex-specific cards (for boys and girls, 
respectively. Personality functions pulled 
by each picture are coded parenthetically 
as follows: interpersonal relate ons (IP, 
aggression (AGG), anxiety/depression (ANX), 
achievement motivation (AM), delay of gratification 
(DG), self concept of competence (SCO), sexual 
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identity (SI), moral judgement (MJ), reality 
testing (RT). 
ADMINISTRATION AND SCORING 

After establishing rapport with the examinee, the 
examiner introduces the test with the following instructions: 
"I have several interesting pictures that I am going to 
show you," look at the persons and places in the pictures 
and tell me a complete story about each picture, one that 
has a beginning and an end. The story should answer three 
question: "What is happening in the picture now? What 
happened before? What will happen in the future?" Following 
presentation of a picture, the examiner records the examinee's 
verbal reaction time, the spontaneous time and then the 
total storytelling time. The minimum time for a story 
is two minutes, and the maximum time for a story is typically 
four to five minutes. 

Inquiries are conducted to elicit the identity and 
interrelationships of the characters, identification of 
settings, and the affective state of the main character 
if this information is omitted. The examinee should be 
given the opportunity to tell a complete story in a spontaneous 
manner (i.e., without prompting). A story is complete 
if it relates: (1) the identities and relationships of 
the characters, the setting, and what the characters are 
doing; (2) what the characters did before; (3) what the 
characters will be doing in the future; and (4) what the 
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main character is thinking and feeling upon resolution 
of the antithetical situation. 

Stories are recorded verbatim by the examiner. The 
full 23 pictures are administered either in two one-hour 
sessions, or in one session with a break after one hour. 
If an examinee is given the TEMAS short form, administration 
of the 9 oictures should be completed within 45 minutes 
in one session, generally without interruption. 

Personality functions are scored on a Likert'-type, 
four-point scale as follows. A score of "1" indicates 
the presence of a highly maladaptive resolution of conflict. 
For example, themes of murder, rape and assult are always 
scored "1" for interpersonal relations, aggression and 
moral judgement. A suicidal theme earns a "1" under the 
anxiety/depression function. The decision to drop out 
of school or steal rather than work results in a "1" for 
achievement motivation and delay of gratification. The 
anticipation of complete failure and concomitant refusal 
to attempt a given task results in a "1" for self concept 
of competence. A character who changes sexes or rejects 
his or her gender earns a "1" in sexual identity. Scores 
of "1" in moral judgement reflect a total lack of regard 
for the consequences of antisocial behavior. Severly impaired 
reality testing would be scored only for the most bizarre 
and impossible resolutions (e.g., inanimate objects come 
alive and kill; a child causes harmful events to occur 
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by a strange power of the mind). A score of "2" for any 
personality function reflects a moderately maladaptive 
resolution. For example, children cheat and get away with 
it; a conflict is resolved by fighting; money is squandered 
rather than saved; homework is avoided in favor of play; 
a child runs away form home and never returns; the monster 
in a dream could also be in the backyard. A score of "3" 
represents a partially adaptive resolution. Here, for 
example, the children who cheat are caught and punished; 
fighting ceases in favor of compromise; money is saved 
for a time and then spend; homework is grudgingly completed; 
a runaway child returns home; the monster does not come 
to life as feared. A score of "4" represents a highly 
adaptive resolution. The child must perceive the intended 
conflict and solve the problem in a mature, viable manner. 
There is implicit in a score of "4", a striving for the 
greater good, sense of responsibility and an intrinsic 
motivation. Here for example, a child rejects the notion 
of cheating as contrary to learning; conflicts are discussed 
and compromises reached; money is saved for the future; 
homework is completed because good grades are valued; a 
child decides to talk to parents rather than run away; 
dreams are never real. 

In addition to personality functioning, TEMAS is also 
scored for a number of descriptive indices: number of 
unanswered inquiries, reaction time (sec), and total storytelling 
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time (min), and verbal fluency (word count). Perceptual 
style is scored for fragmentation, elaboration, omission, 
and transformation of perceptual details. Fragmented responses 
are such that the subject's primary focus is on an isolated 
part of picture (e.g., main character's arm), as opposed 
to a holistic focus. Elaboration refers to whether stories 
tend to be restricted in context in a perseverative theme. 
Omissions and transformations of perceptual detail in the 
pictures is scored when the details are related to the 
identity of perceptual of main and secondary characters, 
event and setting. Affective functioning is scored with 
respect to the emotional state attributed to the main character 
at the end of the story after resolution of the conflict. 
The main character's affect is recorded as happy, sad, 
angry, fearful ^ neutral and/or ambivalent. In addition, 
the congruence of this affective state with the content 
of the story is noted. Descriptive indices are tabulated 
by summing frequency of occurrence across pictures (Costantino, 
Malgady, & Rogler, 1987-in press). 
PRELIMINARY STUDIES 

A pilot study was conducted with a group of eight 
children ranging in age from 6 to 12. Each child was asked 
to tell a story about each picture, responding to the questions: 
What is happening? Who are these characters? Are they 
related? What are they saying? Where are these people? 
Children were retested after a four-month interval to establish 
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test-retest-reliability. Only those pictures which reached 

a .80 inter-child agreement and .90 test-retest stability 

on all five questions were retained (Costantino, 1978). 

A revised set of cards were developed, resulting in a reduction 

to 23 pictures, 12 for both sexes and 11 sex-specific (Costantino, 

19e6). 

The TEMAS pictures were structured to pull nine specific 
personality functions (which are rated on a 4-point scale) 
based upon the nature of psychological conflict confronting 
the main character of the pictures: interpersonal relations, 
aggressions, anxiety and depression, achievement motivation, 
delay of gratification, self concept of competence, sexual 
identity, moral judgement and reality testing. Accordingly, 
a study was conducted to assess the concordance among a 
sample of 14 practicing school and clinical psychologists, 
recruited from community mental health centers and public 
schools in New Yorl: City. Psychologists averaged nearly 
8 years experience in testing and counseling minorities 
and 9 year experience in administering projective techniques. 
With respect to ethnicity, seven were white, one black 
and six were Hispanic. Clinical orientation of the psychologist 
include eclectic, analytic/dynamic, cognitive, and system. 
The psychologist were presented the TEMAS pictures and 
asked independently to indicate which, if any, of the nine 
personality functions were pulled by each picture. They 
also were given the liberty of suggestion pulls other than 
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the nine listed by the author. Results indicated substantial 
concordance among clinicians regarding the pulls of TEMAS 
pictures, ranging from 71-100% agreement. These results, 
then served to define and corroborate the specific personality 
functions to be scored with each picture. 

Since verbal fluency if the sine qua non of validity 
with projective techniques (Anderson and Anderson, 1955), 
two preliminary studies were conducted to compare minority 
examinees' articulateness on TEMAS versus the TAT. In 
the first study, Costantino, Malgady, and Vazquez (1981) 
administered six TAT and minority TEMAS pictures to fourth 
and fifth grade Hispanic children. Results of this study 
indicated that Hispanic children were significantly more 
verbally responsive to TEMAS pictures that TAT pictures, 
and this effect was more enhanced for females than males. 
Consequently, a similar study was conducted (Costantino 
and Malgady, 1983) with a larger number of TEMAS and TAT 
pictures to enhance internal consistency realiability of 
the data, broadening the age range of subjects (grades 
K'-e), comparing both Hispanics and blacks to a white examinee 
group. The results of this second study corroborated the 
sex differences in verbal fluency reported by Costantino, 
Malgady, and Vazquez (1981), and also showed nhat Hispanics 
and Blacks were more verbally fluent on TEMAS compared 
to the TAT. Thus, these findings contradict the notion 
that minorities are deficit in verbal fluency on projective 
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tests r and also provide a cornerstone for the potential 

validity of TEMAS. 

PSYCHOMETRIC PROPERTIES OF TEMAS 

Previous research established an objective consensus 
for scoring TEMAS protocolSr temporal stability of thematic 
contents ana that minority examinees are more articulate 
on TEMAS relative to the TAT. The next study investigated 
the reliability of scoring TEMAS pictures and potential 
difference as a function of sexr age and SES. The subjects 
were 73 Puerto Rican students attending grades K-6 in public 
schools in New York City. The mean age of the subjects 
was 9.19 years. With respect to the occupational scale 
of Hollinghead' s Index of Social Position ^ subjects were 
from low to lower-middle class families. According to 
teacher and parental reports ^ subjects were not undergoing 
psychotherapy for personality disturbance ^ nor presenting 
significant behavioral problems in school. TEMAS was administered 
to subjects with the 23 pictures presented in random order 
by bilingual Hispanic examiners. All subjects were tested 
individually by graduate psychology students in two testing 
sessions conducted in the public schools. Hispanic subjects 
were tested in their dominant language. Subjects subsequently 
responded by tailing a story about each picture for typically 
two to four minutesr which was recorded verbatim by the 
examiner. In the case of subjects who responded in Spanish, 
after being recorded in Spanish, stories were translated 
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into English. All tests were scored by bilingual graduate 
psychology interns, who were blinded to subjects' demographic 
background. 

Internal consistency reliability of TEMAS indices 
was estimated by computing coefficient alpha. Reliability 
of personality functioning indiced was computed only on 
particular pictures intended to "pull" specific functions. 
Pictures pulling interpersonal relations, aggression, and 
moral judgment were highly internally consistent (.72-. 92), 
whereas anxiety/depression achievement motivation, delay 
of gratification, self-concept of competence, sexual identity, 
and reality testing evidence much lower reliabilities (.45-65). 
In P'lrt, reliability of the latter functions may be attributed 
to attenuation by *test length"; that is, scores are based 
on relatively few (3-8) TEMAS pictures. Interrater reliability 
in scoring TEMAS protocols was investigated by comparing 
two independent raters' evaluation of a randomly selected 
sample of 27 subjects. Examinees' protocols were rated 
twice and the two scores were correlated for each picture. 
Correlations were low to moderate for reality testing (r 
=.32 - 60), sexual identity (r = .32 -.36), and achievement 
motivation (r =^ .20 -.65), and substantially higher for 
the remaining functions (median rs = .50 - .68). TEMAS 
indices werft correlated with subjects' age sex (dummy coded), 
and socioeconomic status (SES). By and large, the patterns 
of correlations observed suggested that TEMAS scores are 
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virtually independent of age, although statistically significant 

(p = .05), but low negative correlations exist between 

age and aggression (-.26), and sexual identity (-.34). 

All correlations with sex were negligible. Finally, SES 

had no significant influence on TEMAS scores, possibility 

due to its restricted range. 

A follow-up study was conducted with sample of examinees 
screened for presenting severe behavioral problems in school, 
prior to their referral for psychotherapeutic treatment. 
This study sought to gather evidence of validity, both 
concurrently and predicting psychotherapeutic treatment 
outcomes. The subjects in this study were recruited from 
grades K-3 in New York City public schools. Of nearly 
900 students screened by teacher rating, 210 (120 male, 
90 female) falling below the median rating were solicited 
for participation in the study, and for subsequent referral 
to participate in a comparative study of psychotherapeutic 
treatment modalities (Costantino, 1979). Subjects' families 
were from low to lower-middle class, based on the occupational 
scale of Hollingshead's Index. Subjects were administered 
TEMAS pictures as in the prior study. In order to determine 
the relationship of TEMAS personality assessment to intellectual 
functioning, subjects were also administered three subtests 
of the WISC-R (in Spanish, Escala De Inteligencia Para 
Nino s ) : Vocabulary, Comprehension, and Similarities . 
Foi the purpose of establishing concurrent validity, subjects 
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were administered four criterion tests, and their adaptive 
behavior in experimental role-playing situations was observed 
and rated by examiners. 

Each TEMAS index was correlated w' th subjects' sex; 
(0 = male)r (1 = female); age, and WISC IQ (vocabulary, 
comprehension and similarities subtests). There were no 
significant sex nor IQ effects on TEMAS indices, but age 
effects were low to moderate (r_s = .16 to .37). Results 
of multiple regression analyses indicated that TEMAS profiles 
significantly (P < .05) predicted ego development (sentence 
completion test) (R = .39), mothers' behavior ratings (R 
= .38), and teachers 'behavior rating (R - .49); and observations 
of delay of gratification (R = .32), self concept of competence 
(£ = .50), disruptive behavior (R = .51), disruptive behavior 
(R = .511), and aggressive behavior (R = .32). However, 
there was no significant multiple correlation with respect 
to trait anxiety. Thus, multiple regression coefficients 
ranged from .32 to .51, providing moderate evidence of 
concurrent validity for TEMAS profiles with respect to 
these selected criterion variables. 

In an effort to evaluate the utility of TEMAS profiles 
for predicting psychotherapeutic treatment outcomes, a 
random sample of 298 subjects were randomly assigned to 
three therapeutic interventions (Costantino, 1979). Criterion 
tests and role-playing situations were administered again 
after 20 weekly treatment sessio s. Hierarchical multiple 

ERIC ::G 



regression analysis was used to test the utility of TEMAS 
profiles in predicting post-therapy scores on the criterion 
measure, independent of pre-therapy scores. 

Multiple regression coefficients for prediction of 
criterion measures and observation ratings were statistically 
significant (p < .05) and high with respect to ego development 
(R = .69), rrait anxiety (R = .64), mothers' and teachers' 
be lavior rating (R = .65 - .71), delay of gratification 
(R = .67), disruptiveness (R = .57), and aggression (R 
= .63). Self concept of competence was not significantly 
related to TEMAS. Further, TEMAS profiles appear to provide 
extremely useful pre-therapeuwic information, in the sense 
that 6-22% of the variance in post-therapy outcomes was 
predictable from TEMAS pretest, independent of criterion 
pretests. Thus, this analysis lends support for the clinical 
utility of TEMAS as a tool for evaluation of psychotherapeutic 
outcomes (Malgady, Costantino, and Rogler, 1984). 

Discriminant analyses were conducted with Hispanic 
(N = 138) and Black (N = 73) outpatients at psychiatric 
facilities and children iix public schools in New York City. 
Results indicated that TEMAS profiles significantly discriminated 
the two groups, within both the Hispanic and Black samples. 
Personality profiles discriminated the two groups with 
89% accuracy for Hispanics and 91% accuracy for Blacks. 
Within the clinical sample, the TEMAS indices did not discriminate 
significantly between different DSM-III classifications 
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of disorders of childhood and adolescence. 

The present study was conducted to standardize and 
validate TEMAS cross-culturally with public and private 
school children in San Juanr Puerto Rico. Children were 
administered TEMAS on two occasions (separated by a two-month 
interval) to establish test-retest- reliability, and three 
criterion measures were administered for the purpose of 
establishing concuvrent validity ( Spielberger ' s State-Trait 
Anxiety Scale; Piers-Harris Self Concept Scale; NIMH Center 
for Epidemiological Studies — Depression Scale). In addition 
the TEMAS was administered to psychiatric outpatients diagnosed 
according to DSM-III as experiencing conduct, anxiety or 
adjustment disorder of childhood and adolescence. Construct 
validity was established by discriminating between the 
psychiatric outpatients and a comparable sameple of nonpsychiatric 
elementary school children. 

METHOD 

Subjects 

The subjects were public and private school students 
(N = 280) and psychiatric outpatients (N = 50) from San 
Juan, Puerto Rico. The school sample consisted of 140 
public and 140 private school students, equally distributed 
with respect to sex, from grades kindergarten to six in 
the San Juan metropolitan area. The public v. private 
school distinction was introduced to ensure variability 
due to socioeconomic status, (SES) which has been restricted 
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to low SES subjects in New York City in previous studies. 
The psychiatric sample consisted of male and female outpatients 
from the University of Puerto Rico Medical Center who were 
diagnosed according to DSM-III as experiencing conduct ^ 
anxiety or adjustment disorders of childhood and adolescence. 

The three groups were comparable in age (M = 8.63^ 
SD = 2.12). However r there were rather dramatic differences 
between the groups with respect to household composition 
and socioeconomic factors. The father was present in the 
household in 89.7% of the private school families^ compared 
to only 59% in the public school and 40% in the clinical 
group. Moreover r 6.7% of the private school families were 
receiving sor.a form of public assistance (e.g.^ food stamps; 
subsidized housing) relative to 65.2% of the public school 
and 60.4% of the clinical families. In terms of socioeconomic 
status^ father's and mother's occupations were higher on 
the Hollingshead scale in the private school groupr M =4.83 
( SD = 1.60) and M = 5.29 (SD = 1.27) r respectively, compared 
to father and mother means of 2.84 {SD = 1.44) and 3.87 
(SD =1.74) in the public school sector and 2.86 (SD - 
1.73) and 3.50 (^ = 2.10) in the clinical group. Moreover, 
mothers and fathers in the private school group were better 
educated, with a mean of about 15.5 years of education, 
compared to both the public and clinical groups with means 
of 9.8 to 10.75 years of edacation. 
Procedures 
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The TEMAS test was individually administered to subjects 
with the 23 pictures presented it. random order by Puerto 
Rican examiners. Examiners were graduate psychology students 
who were trained to administer the projective test. Subjects 
were tested in one two-hour session conducted in private 
rooms in the schools and the clinic. The examiners administered 
TEMAS according to the scandardized instructions described 
earlier. Tests were scored by graduate psychology interns 
who were blinded to subjects' demographic backgrounds. 
In addition^ half (K=140) of the public and private school 
students were retested after a two-month interval for the 
purpose of establishing test-retest reliability. 

In order to assess concurrent validity^ subjects were 
tested with three criterion-related measures: the trait 
scale of Spielberger 's State-Trait Anxiety Inventory ( Inventario 
de Ansiedad Estado Y Rasgo para Ninos )^ the Piers-Harris 
Self Concept Scale (Spanish translation) ^ and the NIMH 
Center for Epidemiological Studies Depression Scale (Spanish 
translation ) . 

RESULTS 

Means and standard deviations on the TEMAS scoring 
indices are shown for each group in Table 1. Scores reported 
are averaged across pictures. For cross-cultural comparison ^ 
means and standard deviations are also reported for a public 
school sample of 167 Puerto Ricans in New York City and 
67 Puerto Ricans from outpatient clinics diagnosed with 
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the same DSM-III classifications. As Table 2 shows, the 
Puerto Rican private school subjects were more fluent, 
exhibited fewer omissions and transformations, and required 

fewer inquiries than public school subjects, who, in turn, 
were superior to the clinical group. This pattern of results 
also was reflected in recognition of conflict, sequencing, 
imaginativeness and inter-character relationships. With 
resoect to affective functioning, the three groups were 
comparable in terms of ascribing happy and ambivalent emotional 
states to the main character; however, clinical subjects 
ascribed more sadness, anger, and neutral affect but less 
fearful and were less congruent with the theme of their 
stories. With respect to personality functioning, the 
private school group revealed more adaptive themes than 
the public school group, who were more adaptive than the 
clinical group except for rated aggressiveness, achievement 
motivation and reality testing. 

Comparison of the Puerto Rican sample with the New 
York sample reveals that the low SES New York subjects 
generally have more positive TEMAS profiles than their 
higher SES (private school) native Puerto Rican counterparts. 
Moreover, with the exception of sexual identity, even the 
New York clinical sample had more adaptive or comparable 
personality functioning profiles than the public school 
Puerto Rican sample. A surprising result vas with respect 
to fluency and total time. In the Puerto Rican samples. 
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the private school group talked for less time than the 
public school group, but they were substantially more fluent. 
On the other hand^ the clinical group required considerably 
more prompting to complete their stories ^ but nevertheless 
spoke less than the standard minimum of two minutes. The 
new York subjects were much more verbally fluent than antive 
Puerto Rican subjects r which is an unanticipated finding 
since a given oration requires more words to communicate 
in Spanish than in English. 
Reli bility 

Internal consistency and test-retest reliability estimates 
of the TEMAS indices are reported in Tables 3 and 4 for 
the publicr private r and clinical groups. Test-retest 
reliabilities are based on a random subsample of N = 140 
public and private school subjects. 

As Table 3 shows ^ the objective scoring indices generally 
exhibit acceptable internal consistency estimates in the 
three groups. Fragmentation and transformations ^ which 
rarely occur r had numerous TEMAS pictures with zero variance ^ 
thus lowering the estimates of internal consistency. With 
respect to the affective indices ^ internal consistency 
reliabilities are moderate to high^ with the exception 
of ambivalence in thft privat<* school group (again because 
of items with no variance; see mean in Table 2). The ratings 
of personality functions exhibited generally lower internal 
consistencies compared to the more objective indices. 
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Self/sexual identity moral judgement and reality testing 
show low internal consistency estimates, but this can be 
attributed to the relatively few number of pictures which 
pull these functions (i.e., these are analogous to 3-6 
item tests). Moreover, the best internal consistency estimates 
were derived from the public school group. The most reliable 
personal functions across the three groups appear to be 
person relations, aggression, and self concept. 

Test-retest reliabilities over a two-month interval 
are reported in Table 4 for the two random subsamples of 
70 public and 70 private school subjects. As Table 4 shows, 
the reliabilities are uniformly low, with the exception 
of the anxiety and reality testing personality functions. 
These reliability estimates compare unfavorably with estimates 
from New York samples of Puerto Ricans when reliability 
was based on content (not ratings of pathology). Since 
different clinicians scored the pre- and post-test protocols , 
these are underestimates of true test-retest stability 
since they are further attenuated by inter-rater reliability. 
C oncurrent Validity 

The concurrent validity of the TEMAS personality function 
profile was evaluated by conducting multiple correlation 
analyses with each of the three criterion measures (anxiety, 
depression, self concept) within each of the three groups. 
In the public school group, only delay of gratification 

was significantly correlated with trait anxiety (R = .27, 



£ < .01); reality testing, achievement motivation, and 
delay of gratification were significantly correlated with 
depression (R = .35, £ < .001; and achievement motivation 
and person relations were significantly correlated with 
.f concept (R = .27, £ < .01). In the private school 
group, self/sexual identity and self concept of competence 
were significantly related to trait anxiety (R = .26, £ 
< .05) and depression R = .24, £ < .05); and moral judgement 
and achievement motivation were significantly correlated 
with self concept R = .22, £ < .05). Due in part to the 
small sample size (N = 50), none of the multiple correlations 
were significant in the clinical sample. Combining the 
public and private school groups resulted in slightly higher 
multiple correlations, with essentially the same pattern 
of significant personality functions as reported above. 
In summary, the magnitude of concurrent validity coefficients 
with respect to paper-and-pencil rating scales is somewhat 
lower in the Puerto Rican samples compared to validity 
reported on New York samples (cf., Malgady, Costantino, 
& Rogler, 1984). 
Construct Validity 

A two group (public/private school vs. clinical) discriminant 
analysis was conducted using TEMAS profiles to predic^ 
clinical vs. nonclinical status. The nine TEMAS personality 
functions significantly discriminated the two groups, F 
(9, 321) = 3.52, £ < .001, explaining a modest 9% of the 
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between-group variance. Univariate ^-tests indicated that 
person relations, anxiety, delay of gratification, self 
concept of competence, self /sexual identity, and moral 
judgement significantly differentiated the two groups. 
The strongest discriminators were person relations and 
anxiety (£s < .001). Classification analysis revealed 
that 69% of the subjects were correctly reclassified into 
their original groups on the basis of TEMAS discriminant 
functions. Thus, the results of this analysis lends some 
support to the construct validity of the TEMAS personality 
profile as a gross discriminator of clinical vs. nonclinical 
status . 

DISCUSSION 

The TEM/vS test (Minority Version) was originally developed 
as a clinical tool, presenting ethnically familiar characters 
in urban and fantasy settings, in order tc facilitate minority 
children's identification with the stimuli and thereby 
enhance verbal fluency and self disclosure. Earlier studies 
with New York City Urban examinees indicated that Hispanic 
and Black children are more verbally fluent in telling 
stories aboutTEMAS stimuli than the TAT (Costantino, Malgady, 
& Vazques, 1981; Costantino & Malgady, 1983), thus establishing 
a potential for valid personality assessment of minority 
children. Other studies established the reliability of 
TEMAS, some rudimentary evidence of concurrent validity, 
and clinical utility for predicting psychotherapeutic treatment 
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outcomes (Malgady, Costantino, & Rogler, 1984). 

There is a need to create cross-cultural norms for 
projective tests, as advocated recently by Exner and Weiner 
(1982) and Dana (1986). This issue is addressed by the 
standardization of TEMAS on Hispanic and Black children 
in the United States, and by our recent effort to collect 
data on native Puerto Rican children. Similar work is 
provided by Avila Espada (1986) who has developed a comprehensive 
scoring system for the TAT, which assesses interpersonal 
relationships, aggression, depression, and achievement 
motivation, among other variables. 

The TEMAS test was recently published in two versions, 
a minority form for Hispanics and Blacks and a parallel 
form for whites (Costantino, 1986). Results of the present 
study lend support to the use of TEMAS with examinees in 
Puerto Rico. Reliability and validity estimates for the 
Puerto Rican samples, however, were less compelling than 
for the mainland norm groups. Further, discrimination 
of clinical and nonclinical examinees suggests less clinical 
utility (69% accuracy) than in other mainland studies (89% 
accuracy). Further analyses of the TEMAS stimuli are currently 
in progress to examine characteristics of the stimuli on 
a card-by-card basis. Preliminary evidence suggests that 
some of the TEMAS cards do not pull the designated personality 
functions as consistently in the Puerto Rican samples. 
This seems to be associated with the discrepancy between 
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the urban settings depicted in a New York environment compared 

to urban and rural settings in Puerto Rico. Thus, further 

analyses will be conducted to determine whether a reduced 

set of TEMAS stimuli will demonstrate more acceptable reliability 

and validity, by eliminating cards which behave differently 

in the Puerto Rican environment. 

Mental health clinical services research on Hispanics 
has been organized according to a temporal sequence of 
five interlocking phases (Rogler et al, 1983): the emergence 
of mental health problems; utilization of clinical services; 
assessment and diagnosis; psychotherapeutic intervention; 
and community rehabilitation. Inadequate epidemilolgy 
of mental health needs, underutilization of community mental 
health resources, culturally biased assessment practices, 
culturally insensitive treatment modalities, and lack of 
follow-up services are formidable barriers hindering the 
mental health care of Hispanics. At the hub of this necwork 
of barriers looms the problem of reliable and valid personality 
assessment and psychodiagnosis , since such evaluation will 
shape the nature of epidemiological findings and the course 
of subsequent treatment efforts. Although the TEMAS test 
is still rudimentary in development, the technique is supported 
by promising psychometric findings on reliability and validity. 
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Table 2 



Means and Standard Deviations on TEMAS Indices for Native Puerto 

and New York City Puerto Rlcan Public School 



Rlcan Public, Private and Clinical Samples 
and Clinical Samples 



Index 


PR Public 
(N » 140) 
M SD 


PR Private 
(N - 140) 
M SD 


PR Clinical 
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Table 2, (Cont'd.) 
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PR 
(N 
M 


Private 
= 140) 
SD 


PR Clinical 

(N = 50) 
M SD 


NY Public 
(N = 167) 
M SD 


NY Clinical 
(N » 67) 
M SO 


Iiaaglnatlon 


.10 


.16 


.05 


.12 


.09 


.14 


01 


. uo 


A1 


A^ 

.Uo 


Relationships 


.01 


.02 


.00 


.02 


.00 


.00 


.02 


06 

. \J\J 


09 
. UZ 


A/. 


Happy 


.40 


.24 


.43 


.22 


.40 


.21 


47 




. J J 


oo 
• ZZ 


Sad 


.28 


.19 


.28 


.15 


.39 


.18 


. JO 


99 

. zz 


OQ 


OA 

.20 


Angry 


.09 


.10 


.08 


.09 


.11 


. J- J 


. 1 / 


5 


.11 


• 12 


Fearful 


.08 


.10 


.10 


.11 


.07 


07 


. u/ 


1 A 

. iU 


• 09 


.09 


Neutral 


.13 


.25 


.10 


.19 




. ZD 




.08 


.06 


.16 


Ambivalent 


.01 


.04 


.01 


.03 


.02 


OS 


. Ul 




.01 


• 04 


Congruence 


.06 


.09 


.03 


.06 


.07 


1 1 
• J. J. 




. uo 


A C 

.05 


.12 


Person Relations 


2.52 


.22 


2.67 


.29 


2.A5 


20 


Z . O / 


91 


2.49 


.33 


Aggression 


2.06 


.22 


2.31 


.39 


2.14 


• Z J 


9 7ft 


. zU 


2.44 


.38 


Anxiety 


2.27 


.29 


2.48 


.37 


2.16 


.22 


9 QO 


1 ft 


O 0 0 
Z . J J 


.35 


Ach. Motlv. 


2.80 


.29 


2.9?. 


.30 


2.80 


28 




9 «; 

• ZD 


O *70 

z. /9 


.35 


Delay of Grat. 


2.66 


.40 


.?.76 


.44 


2 55 


• * » (f 


9 70 


O 0 

• zJ 


2.62 


.52 


Self Concept 


2.70 


.47 


2.89 


.50 


2.63 


.46 


3.00 


.46 


2./2 


.33 


Self /Sex. Ident. 


' .80 


.42 


2.83 


.55 


2.64 


.48 


3.40 


.51 


2.50 


.52 


Moral Judgment 


2.32 


.31 


2.56 


.39 


2.29 


.30 


2.91 


.36 


2.52 


.34 


Reality Test. 
« 


2.45 


,54 


2.77 


.50 


2.47 


.37 


2.73 


.65 


2.62 


.62 



Table 3 



Internal Consistency (Alpha) Reliability (r) of TEMAS Indices for Public, Private 

and Clinical Groups 



Index Public Private Clinical 



Inquiries 


.89 


.95 


95 


Time 


.96 


.95 


9S 


Total Omissions 


.87 


.76 


80 


Total Trans. 


.80 


.51 


50 


Fragmentation 


.26 


.31 




Elaboration 


.86 


.94 




Main Char. Om. 


.76 


.69 


.85 


Main Char, Tr. 


.58 


.22 


.76 


Sec. Char. Om. 


.69 


.56 


.72 


Sec. Char. Tr. 


.88 


.79 


.90 


Event Cm. 


.87 


.71 


.86 


Event Tr. 


.51 


.51 


.27 


Setting Om. 


.64 


.53 


.49 


Setting Tr. 


.52 


.18 


.89 


Con.!llct 


.86 


.82 


.87 


Sequencing 


.81 


.80 


.87 


Imagination 


.90 


.92 


.85 


Rela t lonsh Ips 


.13 


.33 




Happy 


.88 


.83 


.83 


Sad 


.81 


.71 


.73 


Angry 


.65 


.68 


.77 


Fearful 


.79 


.76 


.58 


Neutral 


.96 


.93 


.96 


Ambivalent 


.70 


.25 


.83 




Table ?, cont*d. 



Index Public Private Clinical 



Congruence 


.78 


.73 


.70 


Person Relations 


.87 


.78 


.75 


Aggression 


.78 


.69 


.77 


Anxiety 


.75 


.54 


.48 


Ach. Motlv. 


.79 


.38 


.53 


Delay of Grat. 


.65 


.49 


.62 


Self Concept 


.82 


.81 


.92 


Self/Sex Identity 


.61 


.45 


.49 


Moral Judgment 


.31 


.25 


.46 


Reality Test. 


.58 


.36 


.95 



o 

ERIC 



42 

17 



Table 4 

Test-Retest Reliability of Personality Functions for Public (N70) and Private 

(N=70) Random Subsamples 



Personality Function Public Private 

Person Relations .34 .34 

Aggression .09 .18 

Anxiety .59 .57 

Achievement Motivation .36 .33 

Delay of Gratification .19 .06 

Self Concept .20 .36 

Self/Sex Identity .29 .29 

Moral Judgment .30 .36 

Reality Testing .53 .48 



ERIC 



43 

18 



